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Abstract 

Tlois article is a response to ‘Mapping educational research and its impact on 
Australian schools’, Chapter 2 of The Impact of Educational Research, in which 
researchers Allyson Holbrook, John Ainley, Sid Botirke, John Owen, Philip McKenzie, 
Sebastian Mission and Trevor Johnson report on their Commonwealth Education 
Department commissioned study. Tloey mined the Australian Education Index and the 
Bibliography of Education Tloeses in Australia for patterns in education research in 
Australia over the years 1984-1997 and compared the results with additional data 
obtained from university education faculties, postgraduate students of education, 
school principals, system-level administrators and professional associations. 

Tlois response to the study argues that its strength is in the construction of a conceptual 
framework for research as well as in its use of the AEI and BETA. Its framework has 
potential to provide valuable ongoing data to service the whole education community. 
Tloe data garnered through surveys of postgraduate researchers and stakeholder 
groups indicate an interest in research by the education community, but are 
insufficient to provide many answers about the influence of research on schooling in 
Australia. 

Tloe framing of a research question in terms of ‘impact on Australian schools ' 
perpetuates a separation of research community, policy community and school 
community, even though the research framework proposed by the researchers largely 
avoids the separation. In the end, by avoiding a detailed analysis of what might be 
encompassed by a critical construction of either ‘collaboration’ or ‘dissemination’ and 
working uncritically within the given construct of ‘impact’, the researchers lost an 
opportunity to develop a notion of community, ‘collective intelligence’ or public 
interest in educational improvement through research. If dissemination were framed 
in more organic language - in terms of shared knowledge and experience, 
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commitment to understanding and ‘collective intelligence’ - the framework proposed 
by the study could be of major benefit to schooling in Australia. 

‘Mapping educational research and its impact on Australian schools’ (Holbrook et al 
2000), Chapter 2 of Hoe Impact of Educational Research, reports on a project within 
which there are a number of sub-studies. In order to focus on the whole rather than 
the parts of a study that was, to say the least, ambitious in its objectives, this paper is 
organised under the headings ‘what this study does’, ‘what this study does not do’ 
and ‘what this study might have done’. It is hoped that, in this way, the author might 
do justice to the achievements of this ambitious and important set of studies while at 
the same time raising some limitations, along with some of the issues that appear to 
fall within the stated and demanding objectives of the study but on which the study 
is silent. The study deals with issues of some significance to the education community 
in the current decade. This response would like to build on its work by identifying 
some areas of major contribution and some gaps that could be narrowed by taking 
this work forward. 


What this study does 

The study is broad, neatly conceived and yields some useful data. The study 
constructs a framework for analysing research trends, maps educational research in 
Australia in 1984-1997 using the Australian Education Index, supplements this with a 
‘snapshot’ of research in university education faculties in 1997, surveys and analyses 
postgraduate students’, professional associations’ and principals’ perspectives on 
educational research impact and obtains policy makers’ perspectives, drawing 
conclusions based on all sets of data. 

The study is carefully constructed to draw on existing data but, more importantly, to 
establish a methodology and datasets that can be continued after the study, thereby 
creating the much-needed possibility of cumulative knowledge. This is a timely and 
constructive response in a climate of increasing demand for educational research 
results without a parallel resourcing strategy. The continued application of this study’s 
methodology is designed to monitor, as the report puts it, ‘the outputs of Australian 
educational research’. 

The study has a number of components: 

• a conceptual framework for analysing research impact 

• a mapping exercise based on an analysis of the Australian Education Index (AEI) 
1984-1997, the Bibliography of Educational Theses in Australia (BETA) and a 
parallel survey of education faculty research 
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• three surveys, one each of postgraduate students in education, teacher 
professional associations and principals 

• focus groups conducted with system-level administrators 

• analysis of all data sets. 

While each of these components yields interesting and useful data, the richest data is 
from the Australian Education Index. The survey of postgraduate students is 
interesting because of the large sample (although the authors express disappointment 
at the return rate: 41 per cent of distributed surveys) and the high proportion of 
educational research effort represented by postgraduate study. Perhaps the most 
useful aspect of the study, however, is the conceptual framework and the 
methodology for analysing the AEI and BETA. We should know what is being 
systematically investigated in Australian education. Such an analysis is a credible 
starting point. The study reported here points to areas where AEI coverage could be 
improved (books and conference papers) and identifies routine data collection in 
university education faculties as an area where standard procedures would yield 
returns. 

Findings 

The study finds that there has been an overall growth of educational research in the 
period 1984-97 and that priorities for research appear to have been in the category 
of educational processes and structures along with the key learning areas (with a 
growth in the areas of maths, English and science and a decline in history, geography, 
the creative arts and languages other than English). The context of education, schools 
and relationships (particularly community, society and government) are well 
represented and there is a steady but small interest in workplace education and early 
childhood education. There are some ‘signals’ that research methods are becoming 
more diverse, with interpretive and participatory studies becoming more frequent. 
These approaches appear to be welcomed by school-level personnel, while system- 
level administrators are reported to be looking for larger-scale, quantitative research 
with generalisable findings to support policy reform. 

All groups surveyed value research and the study comments (p206) on conditions that 
support its utilisation. The finding of uniformity of educational research interest across 
Australia bodes well for commonality in teaching and policy practice but raises 
questions about depth and research specialisation. 

Insights 

A number of valuable insights arise from this study, some conceptual, some practical 
and some strategic. 
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Conceptually, the four models of ‘systematic educational enquiry’ and the preference 
for that term rather than ‘research’ deserve to be debated within the profession. Much 
could be done with the study’s set of models if we are to take seriously the issue of 
dissemination of research. Standard and agreed classification and organisational 
structures are basic requirements of research accessibility and wide usage. The models 
generated by this study deserve serious consideration. Broad agreement within the 
education community on an enquiry framework with interrelationships between 
categories would assist all members to generate, evaluate and incorporate research. 
The models identified are well conceived and usefully described. 

Practically, the study reports a narrowing of the gap between the worlds of educational 
research and practice. The study’s analysis of the interplay between the conceptual use 
and instrumental use of research is important. The residual tension between those who 
wish to make immediate ‘use’ of research and those who argue the importance of 
‘theoretical’ research might be productively channelled using the argument mounted 
here for the need for long-term change in thought patterns (through conceptual 
research) as well as research that leads to direct action. Additionally, the study 
demonstrates how the AEI can yield valuable data while clearly demonstrating that 
data collection from and within education faculties could be improved. 

Strategically, the study raises some interesting issues. The uniformity it reveals in 
education-based thesis topics across Australia and the apparent failure of faculties with 
research specialisations to attract concentrations of students working in those areas 
should trigger some further analysis. This result may, of course, be a good thing, part 
of a ‘level playing field’ and national consistency. It has significant implications, 
however, for educational policy. The education community needs to ask questions 
about the kind of professional education required to sustain the ‘knowledge economy’ 
proposed by successive Australian governments. In a society where ‘research’, 
‘innovation’ and ‘information economy’ are major policy, program and resourcing 
catalysts, the relationship of university research specialisations and concentrations of 
postgraduate study are matters of strategic importance. 

A second strategic issue is the need for data analysis and research skills in schools. 
While all groups surveyed appeared to support ‘user-oriented action’ as a form of 
research in schools there is concern amongst most groups about the quality of what is 
currently being undertaken. The study asserts: 

While external expertise can provide support, issues regarding the 
quality of data management will only be solved, when schools have staff 
with data collection and analysis skills, and time allocation to undertake 
school-based research and evaluation. (Holbrook et al 2000, p. 197) 
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It goes on to conclude: 

It appears to be unrealistic to expect administrators and teachers in 
schools to keep up to date with research in its basic form. There is 
clearly a role for education systems to provide as many opportunities 
and outlets for teachers to have access to ‘refined’ uses of systematic 
enquiry of the more ‘scientific’ kind. (p. 197) 

It is unclear whether the authors are suggesting schools need specialised staff and 
time for research or whether all teachers require these skills. Nor is it clear what 
constitutes ‘opportunities and outlets’. The whole thrust of this research report 
suggests a community-wide approach to research rather than isolating a role for 
education systems (which seems here to mean employing authorities). It seems 
obvious, and consistent with policy debate generally (Huberman 1987) that 
coordinated and collaborative action will be essential to bridge the gaps identified by 
this research. The study, however, fails to examine or alert the reader to implications 
inherent in some of its data. There are many questions to be asked, for example, 
about the relationship of the four models of research identified by the study and 
factors such as schooling culture, behavioural change, research funding, teaching and 
learning, particularly when the surveys and focus groups appear to reveal both an 
attraction to, and a distrust of, user-oriented action - the most common form of 
research in schools. 

A third strategic issue is the relationship between postgraduate study and research- 
driven change in schools. The observation that postgraduate theses constitute the 
largest body of research in education and that most postgraduate students are existing 
teachers or system-level administrators pursuing professional interests largely 
determined by the individual is an important consideration for the education 
community. As the report points out, this overlap is a potential mechanism for 
integration of research findings into practice. Taken together with the study’s statistic 
that only one percent of salaries in education are research salaries, postgraduate 
research is unlikely to result in either focused effort or transformation of the education 
enterprise without a comprehensive program and resourcing strategy. 


What this study does not do 

The specific research brief is not provided in the report, so the reader must rely on 
the summary of specific objectives. The work ‘includes mapping the dimensions of 
research activity in education, investigating the research contribution of the various 
strands of university activity, and exploring dissemination and utilisation’ (Holbrook 
et al 2000, p. 36). The study achieves the first two of these objectives and produces 
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useful data and observations. It is thin on the exploration of dissemination and 
utilisation. 

In the section ‘specific objectives of the brief, the authors indicate that they 
organised the elements of the study into three groups: ‘mapping the research field 
and the relative contribution of university faculties of education’; ‘research 
outcomes and applications’; and ‘securing optimal impact’. Within the first group, 
the authors report their expectation that the study would ‘assess the strengths and 
weaknesses of educational research in Australian universities’. Under ‘research 
outcomes and applications’ the study was intended to ‘describe and evaluate the 
impact of the outcomes of educational research on the practice of teaching and 
learning in schools, as well as educational policy and administration; and evaluate 
the impact and performance in terms of MCEETYA’s Common and Agreed National 
Goals for Schooling’ (page 38). 

The brief is more explicitly reported in relation to the third area, ‘Securing optimal 
impact’. Here the report lists four requirements of the study: 

• assess the extent of collaboration between educational researchers and the 
teaching profession and pathways for disseminating research finding and 
outcomes; 

• assess the opportunities for priority setting and coordination mechanisms in 
research planning to improve the flow-on benefits from educational research 
to practice; 

• examine the effectiveness of the discipline field (field of education) in meeting 
the needs of practitioners and policy makers; and 

• consider the implications for educational research of emerging modes of 
delivery in teaching practice. 


It seems likely that the brief was seeking more than was reasonable from the 
available resources. The third area requires extensive studies in its own right, far 
more than sample surveys are going to deliver, as does the fourth area, which is 
not explored at all. 

The authors acknowledge some of the problem in their discussion of ‘impact’. 
They place this sensibly in the context of studies and reports in the UK and USA 
as well as linking the current study to previous Australian studies in an attempt to 
make a significant contribution to an already existing body of work, thereby giving 
it a chance of contributing to a more definitive answer in the longer term. What 
they do not acknowledge, however, is that the term ‘impact’ itself is problematic. 
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It is difficult to use the term without a sense of ‘doing unto’. The language implies 
the forceful action of one thing on another, rather than interaction, cooperation or 
collaboration. So although the study presents some useful insights into the 
acceptance of and interest in educational research in schooling, it is limited by the 
assumptions of the policy model within which it was commissioned. 

The schooling community is still conceptualising a research-oriented world. The 
AEI may give us indications of trends in research, enable linkage and isolate gaps. 
The study’s analysis, however, does not tell us about the use of or interest in 
research by the profession. The surveys give us some indication of postgraduate 
student perceptions of published research and its reception in schools. The surveys 
of both professional associations and principals are limited. While 72 professional 
associations of 146 canvassed returned surveys, these represent the views of 
association officers not association members. Certainly, officers of the associations 
would be able to answer questions about the roles of associations vis-a-vis 
research. This needs to be regarded as only a first cut of investigating teacher 
attitudes to and use of research rather than the whole picture. Similarly, the survey 
of principals obtained results from 73 schools, some from the principal’s nominee 
rather than the principal. Secondary schools appear to be over-represented as are 
non-government schools. There is no information provided about the rural-urban 
breakdown of schools. These criticisms do not invalidate the conclusions of the 
report. They do indicate, however, the limits of the data and the need for much 
more work to be done. 

The most disappointing aspect of the report is its comments on dissemination. It is 
true that university faculty could give thought to dissemination in research briefs 
and that the Australian Research Council could require dissemination plans for 
grant acceptance. These measures, however useful, are bureaucratic instruments 
that do not provide coordination, contextualisation or user-orientation. There are 
also inherent dangers in relying on the originators of research to disseminate. 
Given the relationship of postgraduate researchers to practice we would seem to 
run a high risk of unquestioned assumptions and culturally perpetuated bias in 
educational research, not to mention inefficiency, if dissemination relies entirely on 
the same group. More independent and specialised means of ensuring a research 
orientation are required. 

By linking collaboration between researchers and the teaching profession with 
‘pathways to dissemination of research findings and outcomes’ the original brief 
provided an opportunity not taken up by Holbrook et al. In spite of the studies’ 
own findings of many integrated linkages and overlapping roles within the field of 
education, dissemination is viewed in a fairly mechanistic and traditional way. 
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Although this may be understandable in the light of a government emphasis on 
intellectual property and an often narrowly interpreted ‘innovation’ agenda, the 
authors have missed an opportunity to theorise schooling sector knowledge more 
broadly, contextualising it, for example, with work now emerging from bodies like 
the Information Infrastructure Advisory Committee. 

As Nutley et al (2003, p. 6) comment in relation to public policy research in the United 
Kingdom: 

Much effort has gone into improving the dissemination process and 
good practice guidance abounds ... This has developed our 
appreciation of the fact that dissemination is not a single or simple 
process, different messages may be required for different audiences at 
different times. It appears that the promulgation of individual research 
findings may be less appropriate than distilling and sharing pre-digested 
research summaries. Evidence to date also suggests that multiple 
channels of communication - horizontal as well as vertical; networks as 
well as hierarchies - may need to be developed in parallel. 

In the context of government interest in a ‘knowledge-based society’ or ‘information 
economy’ (the context of this commission), the schooling sector can be conceived as 
underpinning social capital. If knowledge can be ‘grown’ and ‘developed’ we could 
anticipate the growth of ‘collective knowledge’ and use such a concept to transform 
linear views of a researching, publishing, implementing continuum. Given that, at the 
same time this study was underway, some universities in Australia were attempting to 
restructure their teacher education programs to accommodate such a transformation 
(Reid and O’Donaghue 2001) and place research at the heart of teacher education, 
this is a major opportunity lost. 

Writers like Pierre Levy (1997) argue for the development of ‘collective intelligence’ 
based on the continuous sharing of knowledge by individuals with a commitment to 
learning, common interests, but diverse talents and understanding. In order to achieve 
this, the entire sector must value learning, growth and knowledge - the motherhood 
notions of schooling. In this construct, teachers, as well as policy makers, 
administrators and researchers, must bring an attitude of questioning and observation 
to their work, seeking to ‘know’ rather than ‘assume’ or ‘believe’. They will contribute 
to and draw from the collective, growing ‘intelligence’ of the sector. Collaboration 
becomes a fundamental requirement of twenty-first century schooling, even as 
teaching and learning in the schooling sector becomes a more distributed activity 
through policies of devolution and the application of technology systems. 


10 


COLLABORATION, COMMUNITY AND COLLECTIVE INTELLIGENCE 


If students from at least pre-school age are exposed to a wide range of 
communication media, experiencing what cannot be controlled or even influenced by 
the school, and if knowledge in areas as diverse as brain research, drug therapy, food 
technology, cognitive theory, psychology and sociology are growing substantially in 
relevance to schooling each year, then the sector must achieve what Levy (1997, p. 
13) calls the ‘efficient transformation of difference into collective wealth’. This 
achievement depends on our capacity to work with a range of people and 
experiences - many outside what we think of as the schooling sector - to 
continuously observe, evaluate, understand, adjust, experiment, contribute and 
reconceptualise our own knowledge. 

While this is not fundamentally different to cycles of action research, the imperative 
is greater, with pressure from governments, the public, the profession and the 
knowledge coming from a much broader range of sources, both inside and outside 
education. Thus user-oriented action, while undertaken in a local context, must take 
into account what is happening elsewhere in the present and what has happened 
anywhere in the past. 

In the light of this situation, it is inadequate to conceive of ‘dissemination’ in terms of 
‘broadcasting’ as the study appears to do, as if what is required is bulletins from 
researchers to practitioners. What is required is a recognition throughout the sector of 
the importance of data and information flow, the creation, analysis and recognition 
of ‘knowledge’, and the development of and support for a culture and infrastructure 
of knowledge growth, sharing and application that will serve both the sector and the 
public. This requires detailed analysis and discussion of a number of issues touched 
on in the Holbrook et al study: the relationship of postgraduate study to school 
leadership and renewal, the capacity of schools to analyse their own organisational 
learning, the profile of researchers, conflict of interest in the commissioning of 
research and how research findings are published. We need transparency, checks and 
balances and discussion of governance issues in relation to research. This can be 
difficult when schooling is so heavily funded from the one source - government - 
and particularly when governments have a neo-liberal or instrumentalist bent. 
However, it should be possible to use the public nature of schooling to create an 
open, distributed, transparent and responsive system in which knowledge is shared, 
analysed, contested and grown, where research needs can be continuously identified 
and results acknowledged and widely ‘owned’. 

Again, Nutley et al (2003, p. 6) summarise the issue in relation to public policy: 
‘pushing information from the centre out is insufficient and often ineffective: we also 
need to develop strategies that encourage a “pull” for information from potential end 
users’. 
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Finally, even on its own narrow view of ‘dissemination’, the report does not take its 
own advice. While it is no doubt what the funding agency requested, the report is 
dense, packed with information that in different formats and contexts would be of 
great interest to teachers, school and system-level administrators and policy makers. 
It has no dissemination plan and it is presented in a format that guarantees minimum 
usage. 


What the study might have done 

The study frames useful interactions between three kinds of knowledge: the four 
models of systematic educational enquiry; experiential or craft knowledge; and local 
knowledge. It does not attempt, however, to develop any practical framework in 
which this interplay might be supported. In the last five years there have been a 
number of initiatives around the world looking at making research more accessible 
to the education community and supporting its dissemination. While at least one of 
these, the British Columbia-based Public Knowledge Project, was underway at the 
time Holbrook et al were undertaking this study, most documentation was 
subsequent to the report (Willinsky 1999, 2000, 2001). Others, such as the Campbell 
Collaboration, 1 President Bush’s What Works Clearninghouse, 2 and research networks 
such as BECTA’s (2002) are very recent developments. In the last twelve months the 
Australian government’s Systemic Infrastructure Initiative Higher Education 
Bandwidth Advisory Committee (DEST 2002a) and the Information Infrastructure 
Advisory Committee (DEST 2002b) have reported with substantial recommendations 
in relation to research networks and access to research. 

The most significant of the international initiatives, at least conceptually, is the Public 
Knowledge Project, which conceptualises educational research being published on 
the web, accessible by the general public (in the same way as health information is 
available on the web), with more extensive reports available for those who require 
them, and the opportunity for dialogue between users of the knowledge and 
researchers. In this way, practitioners would be able to contribute findings, 
observations and data directly to researchers in the interests of professional 
knowledge and, ultimately, public knowledge. Technology makes such infrastructure 
and exchange feasible within the public domain. 

The Bush administration’s ‘What Works’ database is a much narrower conception 
fuelling fears of narrow, ‘quick fix’ approaches to complex educational situations. But 
Willinsky (2001) has successfully demonstrated that this does not have to be the 
model accepted either by the profession or government. In Australia, the Ministerial 
Council on Education, Employment, Training and Youth Affairs (MCEETYA) has 
expressed interest in exploring a framework for research accessibility and the 
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MCEETYA ICT in Schools Taskforce is in the process of developing a research strategy 
for ministers. Further research is required to determine how such an infrastructure 
could work, but there are already some steps taken in that direction as reflected, for 
example, in debates about electronic publishing within AARE. 3 If researchers can 
publish online, and we can establish consistent and standard metadata and indexing 
requirements, it would make sense for governments to support educational research 
databases linked to communication systems that enable (1) the public to access plain 
language summaries of educational research, (2) teachers and other practitioners to 
access the latest research findings in designated areas and to report their own use of 
those findings to the original researchers, (3) researchers to publish results quickly 
and to connect with others in the same field and (4) packages of research information 
to be ‘pushed’ to and ‘pulled’ by multiple audiences including the media. 

Achievement of such a framework requires the application of information 
management and communications technologies to research data. The skills to achieve 
this can be found within the education community, broadly defined. 

It is easy with hindsight to identify deficits in the mapping study that, just two years 
later, appear obvious. However, the issue of disseminating or marketing educational 
research features heavily throughout the Impact of Educational Research volume, 
with little consideration of how the deficit might be overcome. Addressing this 
requires a management framework, distributed rather than centralised, yet 
comprehensive enough to reach not only the educational research community, but 
also all the users of that research, including the general public. The AEI is a major 
asset, but only one part of what is now required. A broader conceptualisation is 
required, an approach that is more eclectic, flexible and capable of specific audience 
focus. Why is it that newspapers in Australia routinely carry items about medical 
research but rarely educational research? Why does the Australian Consumers’ 
Association produce a regular Health Reader but not a comparable Education Reader 
or Education Choice ? Part of the answer to that question must lie in access to medical 
research databases with their capacity to generate plain language summaries and 
abstracts. 

The success of services such as AustLII (Australasian Legal Information Institute) 
suggests that the public can be served by quite technical professional databases if 
they are planned and conceived well enough. The study report ignores the 
component of the brief indicating it should ‘consider the implications for education 
research of emerging modes of delivery in teaching practice’ (Holbrook et al 2000, p. 
38). Some of these modes of delivery would have informed the conceptualisation of 
‘dissemination’ of research and the linkages that might free up rigid delineation of 
roles within the sector. 


13 


JULIAN DELLIT 


The conceptual framework outlined by Holbrook et al should be further explored and 
perhaps developed to underpin an Australian public knowledge framework in 
education, linked to similar overseas efforts. This would make a major contribution 
to the bridging of the research-practitioner divide while raising the profile and 
sharpening the edges of educational research. It would make use of the core of 
intellectual work contributed by this study while recognising that some of the 
questions posed by its brief require the intellectual focus of researchers, 
administrators and teachers over a number of years. One of the limitations inherent 
in commissioned research, especially when the commissioning agency is the major 
funder of educational research in Australia, is the inadvisability of the research 
providers raising too many questions about the framing of the commission. Within the 
terms of its commission this report has delivered the groundwork for understanding 
the potential of educational research over time. With funding and a more organic 
framework, others may build on that understanding. 


Notes 

1 Information about the Campbell Collaboration can be found at 

http://www.campbellcollaboration.org/ 

2 Information about the What Works Clearinghouse can be found at 
http://www.ed.gov/offices/OESE/SASA/ww/index.html 

3 See, for example, information on the association website at 

http://www.aare.edu.au/index.htm 
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